UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark OtBce 

Address: COMMISSIONER FOR PATENTS 

APPLICATION NO. | FILING DATE | FIRST NAMED INVENTOR | ATTORNEY DOCKET NO. | CONFIRMATION NO. | 

10/686,240 10/15/2003 Christopher J. C. Burges MS1-1674US 7597 

EXAMINER I 
THOMAS, JASON M 
ART UNIT I PAPER NUMBER | 

2423 

NOTIFICATION DATE | DELIVERY MODE "[ 
08/21/2009 ELECTRONIC 

Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 

Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the 

following e-mail address(es): 

lhptoms@leehayes.com 




United States Patent and Trademark Ofhce 



22801 7590 08/21/2009 

LEE & HAYES, PLLC 

601 W. RIVERSIDE AVENUE 

SUITE 1400 

SPOKANE, WA 99201 



PTOL-90A (Rev. 04/07) 



KJtSiVrXS nvrliyjts OUff Iff fcff Jr 


Application No. 

10/686,240 


Applicant(s) 

BURGES ET AL. 


Examiner 

Jason Thomas 


Art Unit 

2423 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
eamed patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )□ Responsive to communication(s) filed on 19 May 2009 . 
2a )□ This action is FINAL. 2b)|3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-3. 7-13. 15. 16 and 21-27 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) n Claim(s) is/are allowed. 

6) IEI Claim(s) 1-3.7-13.15.16 and 21-27 is/are rejected. 
/)□ Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10)0 The drawing(s) filed on is/are: a)^ accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held In abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 !)□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)n All b)n Some * c)^ None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attach ment(s) 

1 ) ^ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/IVIail Date. 

3) □ Information Disclosure Statement(s) (PTO/SB/08) 5) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date . 6) □ Other: . 



PTOL-T26'(Rev^'o8-0^^ 



Office Action Summary 



Part of Paper No./Mail Date 2009081 3 



Application/Control Number: 10/686,240 
Art Unit: 2423 



Page 2 



DETAILED ACTION 

Response to Arguments 

1 . Applicant's arguments, see pp. 11-19, filed May 19, 2009, with respect to the 
rejection(s) of clalm(s) 1-3, 7-13, 15, 16 and 21-27 under 35 U.S.C. Section 103(a) 
have been fully considered and are persuasive. Therefore, the rejection has been 
withdrawn. However, upon further consideration, a new ground(s) of rejection is 
made in view of Foote et al., U.S. Pat. No. 6,774,917 B1 and Snook, U.S. Pat. No. 
6,400,378 B1. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or deschbed as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1 and 21 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Foote et al., U.S. Pat. No. 6,774,917 B1 (hereinafter Foote), in view of Cobbley et 
al., U.S. Pat. No. 5,818,510 (hereinafter Cobbley), Snook, U.S. Pat. No. 6,400,378 
81 (hereinafter Snook) and Maybury et al., U.S. Pat. No. 6,961,654 81 (hereinafter 
Maybury). 

Regarding claims 1 and 21: Foote discloses a system including an 
executable means for interactive similarity searching, retrieval and browsing (see 
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[col. 6, II. 28-67] for a system which executes similarity searching) comprising: 
receiving a request for information regarding a media object (see [col. 2, II. 25-42], 
[col. 20, II. 18-28] for submitting a query); inferring the information from repeat 
instances of media objects occurring within one or more media streams (see [col. 
20, II. 18-60], [col. 21, II. 9-22] for identifying similar instances of media properties 
using both image frame analysis and annotations, which read on objects, where 
similarity is based on a segment similarity score and said annotations, and where 
the ability to search multiple videos reads on inferring from one or more media 
streams); and returning the information (see [figs. 32, 33 and 34] for returning the 
search results) but while Foote teaches using annotations within the media as an 
additional means to assist in searching for repeat instances (see [col. 20, II. 29-43] 
where annotation are linked to the video segments) and also determining different 
versions of repeating instances (see [col. 3, II. 18-36], [col. 16, II. 7-16] for multiple 
segments which are similar), Foote does not clearly define how annotations are 
used or teach presenting the media objects by comparing temporal lengths of the 
plural instances of the search results such that the different versions of the media 
object are selected from the group comprising: a longest version of the media object; 
a number of longer versions of the media object; a shortest version of the media 
object; and a number of shorter versions of the media object; nor does Foote teach 
wherein the inferring further comprises determining a number of related media 
objects, wherein: (i) the related media objects are determined based on temporal 
proximities of media objects relative to the media object associated with the request. 
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and (ii) the related media objects have a higher frequency of repeat instances 
relative to one another. 

Cobbley teaches a system which also segments media where the media also 
include keywords, which read on annotations, which are used to describe the media 
segment to assist In Indexing and searching (see [cols. 3-4, II. 65-12], [col. 4, II. 43- 
50], [cols. 5-6, II. 56-3], [col.8, II. 4-16]). Therefore, it would have been obvious to 
one of ordinary skill in the art at the time the invention was made to modify the way 
in which text information is used by relying also on pre-embedded text information 
describing the video stream, as taught by Cobbley in order to provide assistance in 
Identifying repeat instances of the media objects. 

Snook teaches a method of presenting media clips to a user by duration such 
that the clips can be displayed by duration from shortest to longest and in ascending 
or descending order (see [col. 4, II. 36-56], [col. 6, II. 13-21] which reads on 
determining a longest or shortest version of the media object). Therefore, It would 
have been obvious to one of ordinary skill in the art at the time the invention was 
made to modify the way in which the search results are presented to the user by 
allowing the media segments (clips) to be sorted by duration, as taught by Snook, In 
order to provide the user with a simplified and effective way of reviewing the search 
results. 

Snook does not teach wherein the inferring further comprises determining a 
number of related media objects, wherein: (i) the related media objects are 
determined based on temporal proximities of media objects relative to the media 
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object associated with the request, and (ii) the related media objects have a higher 
frequency of repeat instances relative to one another. 

Maybury teaches determining related media objects based on temporal 
proximities associated with media object in question and based on media objects 
having a higher frequency relative to another (see [figs. 3, 4, 9], [col. 4, II. 14-25], 
[cols. 6, II. 53-2], [col. 17, II. 45-60] for tracing the temporal proximity of occurrence of 
the associated news story segment and also analyzing the frequency of occurrence 
or words, images, or sounds an indicative of the most common or most important 
content). Therefore, it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to modify the searching means by enabling 
additional searching methodologies such as basing the search on a temporal 
proximity or object frequency as taught by Maybury in order to provide additional 
means of specifying the search interest of the requesting user. 

Regarding claim 13: Foote discloses a system including an executable 
means for interactive similarity searching, retrieval and browsing (see [col. 6, II. 28- 
67] for a system which executes similarity searching) comprising: receiving a request 
for information regarding a media object (see [col. 2, II. 25-42], [col. 20, II. 18-28] for 
submitting a query); inferring the information from repeat instances of media objects 
occurring within one or more media streams (see [col. 20, II. 18-60], [col. 21 , II. 9-22] 
for identifying similar instances of media properties using both image frame analysis 
and annotations, which read on objects, where similarity is based on a segment 
similarity score and said annotations, and where the ability to search multiple videos 
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reads on inferring from one or more media streams); and returning the information 
(see [figs. 32, 33 and 34] for returning the search results) but while Foote teaches 
using annotations within the media as an additional means to assist in searching for 
repeat instances (see [col. 20, II. 29-43] where annotation are linked to the video 
segments) and also determining different versions of repeating instances (see [col. 
3, II. 18-36], [col. 16, II. 7-16] for multiple segments which are similar), Foote does 
not clearly define how annotations are used or teach presenting the media objects 
by comparing temporal lengths of the plural instances of the search results such that 
the different versions of the media object are selected from the group comprising: a 
longest version of the media object; a number of longer versions of the media object; 
a shortest version of the media object; and a number of shorter versions of the 
media object; nor does Foote teach wherein the inferring further comprises 
determining a number of related media objects, wherein: (i) the related media 
objects are determined based on temporal proximities of media objects relative to 
the media object associated with the request, and (ii) the related media objects have 
a higher frequency of repeat instances relative to one another. 

Cobbley teaches a system which also segments media where the media also 
include keywords, which read on annotations, which are used to describe the media 
segment to assist in indexing and searching (see [cols. 3-4, II. 65-12], [col. 4, II. 43- 
50], [cols. 5-6, II. 56-3], [col.8, II. 4-16]). Therefore, it would have been obvious to 
one of ordinary skill in the art at the time the invention was made to modify the way 
in which text information is used by relying also on pre-embedded text information 
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describing tlie video stream, as taught by Cobbley in order to provide assistance in 
identifying repeat instances of the media objects. 

Snook teaches a method of presenting media clips to a user by duration such 
that the clips can be displayed by duration from shortest to longest and in ascending 
or descending order (see [col. 4, II. 36-56], [col. 6, II. 13-21] which reads on 
determining a longest or shortest version of the media object). Therefore, it would 
have been obvious to one of ordinary skill in the art at the time the invention was 
made to modify the way in which the search results are presented to the user by 
allowing the media segments (clips) to be sorted by duration, as taught by Snook, in 
order to provide the user with a simplified and effective way of reviewing the search 
results. 

Snook does not teach wherein the inferring further comprises determining a 
number of related media objects, wherein: (i) the related media objects are 
determined based on temporal proximities of media objects relative to the media 
object associated with the request, and (ii) the related media objects have a higher 
frequency of repeat instances relative to one another. 

Maybury teaches determining related media objects based on temporal 
proximities associated with media object in question and based on media objects 
having a higher frequency relative to another (see [figs. 3, 4, 9], [col. 4, II. 14-25], 
[cols. 6-7, II. 53-2], [col. 17, II. 45-60] for tracing the temporal proximity of occurrence 
of the associated news story segment and also analyzing the frequency of 
occurrence or words, images, or sounds an indicative of the most common or most 
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important content). Therefore, it would have been obvious to one of ordinary sl<ill in 
the art at the time the invention was made to modify the searching means by 
enabling additional searching methodologies such as basing the search on a 
temporal proximity or object frequency as taught by Maybury in order to provide 
additional means of specifying the search interest of the requesting user. 

While these references do not explicitly teach a first, second or third request 
for media it is the combined teachings of the aforementioned prior art teach 
interfaces which are used conducting a search (see Maybury [figs. 32, 33], [col. 20, 
II. 18-28]; see also Maybury [figs. 14-19, 21, 22] for an interface which provides a 
means for indicating a type of search a user should choose to input; see also [fig. 
14, 18], [cols. 6-7, II. 53-2] for the ability to identify the most common, which reads 
on a most popular media object), in addition, the ability present, which reads on 
rendering, the searched repeated and related media segments is also taught in the 
aforementioned prior art (see Foote [col. 3, 47-60]; see also Maybury [col. 5, II. 10- 
20] for interfaces which allow the search results to be presented to the user). 

Regarding claims 2 and 22: The combined teachings of the aforementioned 
prior art, teach wherein the inferring comprises searching a database for the 
information, the database including media objects and records of repeat instances of 
the media objects (see Foote [col. 20, II. 18-43]; see also Cobbley [cols. 6-7, II. 53-3], 
[col. 8, II. 45-64]). 

Regarding claims 3 and 23: The combined teachings of the aforementioned 
prior art, teach wherein the inferring comprises: monitoring the one or more media 
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streams; identifying tlie repeat instances; and storing records of the repeat instances 
in a database (see Foote [col. 20, II. 18-42] where a database contains one or more 
media streams; see also Maybury [fig. 15]; for monitoring one or more media 
streams; Cobbley [cols. 6-7, II. 53-3], [col. 8, II. 45-64] identifying repeat instances 
and storing records of the repeat instances). 

Regarding claims 7 and 25: The combined teachings of the aforementioned 
prior art, teach wherein the inferring comprises matching a key word from the 
request with metadata extracted from a media object (see Maybury [figs. 15, 21 , 22] 
for a keyword search). 

Regarding claims 8 and 26: The combined teachings of the aforementioned 
prior art, teach wherein the inferring comprises matching date and time information 
from the request with date and time information of a media object stored in a 
database (see Maybury [fig. 15] for matching date and time). 

Regarding claims 9 and 27: The combined teachings of the aforementioned 
prior art, teach wherein the inferring comprises limiting returned media objects based 
on constraints contained within the request (see Maybury [fig. 15] where criteria 
such as type of search and data and time limit the returned media objects). 

Regarding claim 10: The combined teachings of the aforementioned prior 
art, teach wherein the inferhng comprises identifying temporal endpoints of each 
repeat instance of the media object (see Foote [cols. 2-3, II. 53-10], [col. 3, II. 47-60] 
where defining a segment is identifying a start and endpoint). 
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Regarding claim 11: The combined teachings of the aforementioned prior 
art, teach wherein the identifying is based on an identifier included in the request, 
the identifier selected from the group comprising: a fingerprint of the media object; 
and a time stamp and channel code associated with the media object (see Foote 
[col. 20, II. 18-43] where an actual video segment, which reads on a fingerprint, is 
used for a search; see also Maybury [fig. 15] where time information, which reads on 
a time stamp, and new source, which reads on a channel code, is used for a 
search). 

Regarding claim 12: The combined teachings of the aforementioned prior 
art, teach a server computer comprising the processor-readable medium as recited 
in claim 1 (see Maybury [col. 5, II. 32-45] for running a search system using a 
server). 

Regarding claim 15: The combined teachings of the aforementioned prior 
art, teach comprising further processor-executable instructions configured for 
rendering a media stream that includes the media object (see Foote [col. 3, 47-60]; 
see also Maybury [col. 5, II. 10-20] for interfaces which allow the search results to be 
presented to the user, which reads on rendering). 

Regarding claim 16: The combined teachings of the aforementioned prior 
art, teach a client computer comprising the processor-readable medium as recited in 
claim 13 (see Maybury [col. 5, II. 32-45] for using a client computer for said system). 

Regarding claim 24: The combined teachings of the aforementioned prior 
art, teach wherein the inferring comprises determining a number of related media 
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objects, the related media objects occurring within a close temporal proximity of the 
media object with a higher frequency of repeat instances relative to one another (see 
[figs. 3, 4, 9], [col. 4, II. 14-25], [cols. 6, II. 53-2], [col. 17, II. 45-60] for tracing the 
temporal proximity of occurrence of the associated news story segment and also 
analyzing the frequency of occurrence or words, images, or sounds an indicative of 
the most common or most important content). 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

• Bolle et al., U.S. Pat. No. 6,675,1 74 - Exact or similar matching of media 
sequences such as short of long versions of commercials, etc. 

• Ahmad et a!., U.S. Pat. No. 6,263,507 - Identifying and displaying related 
news stories based on embedded data. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jason Thomas whose telephone number is (571 ) 270- 
5080. The examiner can normally be reached on Mon. - Thurs., 8:00 a.m. - 5:00 p.m.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Koenig can be reached on (571) 272-7296. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

J. Thomas 
/Andrew Y Koenig/ 

Supervisory Patent Examiner, Art Unit 2423 



